Multilingual Summarization Evaluation without Human Models
نویسندگان
چکیده
We study correlation of rankings of text summarization systems using evaluation methods with and without human models. We apply our comparison framework to various well-established contentbased evaluation measures in text summarization such as coverage, Responsiveness, Pyramids and ROUGE studying their associations in various text summarization tasks including generic and focus-based multi-document summarization in English and generic single-document summarization in French and Spanish. The research is carried out using a new content-based evaluation framework called FRESA to compute a variety of divergences among probability distributions.
منابع مشابه
NII at the 2006 Multilingual Summarization Evaluation
In this paper I detail the implementation of an extractionbased summarization system that uses sentence clustering and named entity identification as main features for the 2006 Multilingual Summarization Evaluation. I discuss some of the failings of my system, and what can be done to improve it.
متن کاملPKUSUMSUM : A Java Platform for Multilingual Document Summarization
PKUSUMSUM is a Java platform for multilingual document summarization, and it supports multiple languages, integrates 10 automatic summarization methods, and tackles three typical summarization tasks. The summarization platform has been released and users can easily use and update it. In this paper, we make a brief description of the characteristics, the summarization methods, and the evaluation...
متن کاملColumbia University at Mse 2005
We describe our participation in the Multilingual Summarization Evaluation 2005. We describe the Columbia summarizers that were used in our submission and discuss the evaluation, drawing conclusions about the performance of our summarizers, discussing the state of multilingual summarization in general and also listing issues that need consideration for future evaluations.
متن کاملMultilingual Single-Document Summarization with MUSE
MUltilingual Sentence Extractor (MUSE) is aimed at multilingual single-document summarization. MUSE implements a supervised language-independent summarization approach based on optimization of multiple sentence ranking methods using a Genetic Algorithm. The main advantage of MUSE is its language-independency – it is using statistical sentence features, which can be calculated for sentences in a...
متن کاملSimilarity-based Multilingual Multi-Document Summarization
We present a new approach for summarizing clusters of documents on the same event, some of which are machine translations of foreign-language documents and some of which are English. Our approach to multilingual multi-document summarization uses text similarity to choose sentences from English documents based on the content of the machine translated documents. A manual evaluation shows that 68%...
متن کامل